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ABSTRACT 


The Department of Defense (DOD) failed in its previous attempt to acquire an 
enterprise electronic health record (EHR) system. The earlier program was plagued with 
schedule delays and cost overruns, which caused its failure. In July 2015, the DOD’s 
Program Executive Office awarded a $4.3 billion contract for a new EHR system that 
was below cost and ahead of schedule. The objective of this research is to investigate the 
key reasons why the DOD has succeeded in acquiring its most recent EHR. 

This study interviewed nine members of the Program Management Office (PMO) 
team for their opinions on and experiences with their acquisition and management 
strategies used during the procurement. The research showed that members from the 
program management to the program executive level shared commonalities in 
management styles that led to the successful acquisition of the DOD’s newest EHR 
system. The research identified several factors key to the program’s success: a tailored 
acquisition plan that allowed the PMO to directly report to the Under Secretary of 
Defense for Acquisition, Technology and Logistics; a separation between the medical 
communities and the acquisition team; an engagement with industry early in the process; 
and a motivated leadership. 
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I. 


INTRODUCTION 


A. PURPOSE 

In February 2013, the U.S. Department of Defense (DOD) and U.S. Department 
of Veterans Affairs (VA) ended their partnership in developing a joint electronic health 
record (EHR) that would allow for interoperability of data between the two departments. 
According to the U.S. Government Accountability Office (2014), the abandoned project 
wasted four years and roughly $1 billion between the DOD and VA. In July 2015, the 
DOD announced it had purchased a new EHR system. According Garamone, the program 
management office (PMO) responsible for the acquisition of the new system acquired a 
commercial product for $4.3 billion in only two years. The contract award of the new 
EHR system was on schedule and below cost (2015). 

The purpose of this study is to examine why the DOD has succeeded in acquiring 
its most recent enterprise EHR system in record time at a significant cost savings when 
the previous attempts had failed. 

B. OVERVIEW 

The DOD has a long history of attempting to acquire an EHR for the longitudinal 
storage and access of healthcare data for active duty service members and their 
dependents. In the 1980s, the DOD implemented its first EHR, the Composite Health 
Care System (CHCS), which initially achieved success. However, future attempts to 
upgrade the CHCS system resulted in numerous acquisition failures (Government 
Accountability Office [GAO], 2002). 

Since the CHCS, several attempts to acquire and modernize the DOD’s EHR have 
been plagued with cost over-mns and delays (GAO, 2010). Several reports by the 
Government Accountability Office (GAO) reveal previous EHR acquisitions were 
impacted by poor project planning, management weaknesses, lack of governance, 
inadequate accountability, and poor oversight, which cost taxpayers billions over the span 
of 10 to 15 years (GAO, 2010). 
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In July 2015, the Program Executive Office (PEO) of the Defense Health 
Management Systems (DHMS) awarded a $4.3 billion contract to purchase a commercial 
off-the-shelf (COTS) EHR. The Defense Healthcare Management System Modernization 
(DHMSM) program did not incur protests from the competition and was able to reduce 
the cost from an estimated $13 billion to an actual cost of $8.3 billion for the estimated 
10-year life-cycle of the implementation (Garamone, 2015). It is unclear specifically 
what the PEO DHMS (hereafter referred to as PEO) did to achieve this success. 

C. RESEARCH QUESTIONS 

This research examines how the DOD’s PEO and DHMSM program succeeded in 
successfully awarding a multi-billion-dollar contract for a DOD electronic health record 
when previous attempts to acquire an EHR had failed. 

Related research questions include the following: 

• What were the key reasons for the failures of previous enterprise EHR 
programs within the DOD? 

• What obstacles and risks did the program office face when implementing 
the new system? 

• What acquisition and managerial strategies directly led to success? 

• What can current program managers of major software acquisitions learn 
from the DHMSM program? 

D. PROJECT OBJECTIVES AND SCOPE 

The primary objective of this research is to investigate the key reasons why the 
DOD has succeeded in acquiring its most recent enterprise EHR in record time and at a 
significant cost savings. More specifically, this study focuses on events from the PEO 
and the DHMSM program office from its creation in June 2013 to the new contract award 
in July 2015. A secondary objective is to explore how and why previous attempts by the 
DOD to acquire an EHR failed to achieve the same success. A final goal is to provide 
feedback and proposals for improvements to the DOD’s acquisition framework to benefit 
other major acquisition programs. 


2 



This research examines the acquisition phase for the programs in question. This 
research does not explore the implementation or deployment of the systems. In Chapter 
II, the costs and other implementation concerns are documented for each previous DOD 
EHR program. This is to compare the relative effectiveness of their corresponding 
acquisition processes. This research does not address any future activities or 
implementation plans from the PEO or the DHMSM program management offices. 

E. METHODOLOGY 

This study used interviews with key people from the PEO and the DHMSM 
program office. These people had direct experience with and decision authority in the 
program. The interview format enabled each person to tell his or her story directly and in 
detail, and by integrating the stories of multiple key people, this research was able to 
weave together a rich and coherent understanding and set of lessons learned. From the 
PEO office, two executives were interviewed. From the DHMSM program, three 
program managers were interviewed, and within the PEO and DHMSM program, four 
key managers were interviewed. 

This intent of this section is to show the research organization and procedures. 

1. Subjects 

Each interview subject was selected based on his or her position and influence in 
the program. With the support of co-advisor Mark Krause, who is familiar with the 
program office and personnel, a cohort of 12 individuals was identified as possible 
research subjects. A total of nine subjects volunteered to participate in the research. 
When the first nine subjects were selected, the researcher did not pursue or consider any 
additional research candidates. Research participants were checked by the student 
researcher against the PEO and DHMSM organizational charts for their positions and 
roles during the timeframe in question. The researcher also submitted the list of potential 
participants to Mark Krause for approval before providing the research candidate a 
consent form. 
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2 . 


Consent 


All research subjects were provided a consent form prior to their participation in 
the research. The researcher requested that the subjects read and understand the research 
consent process prior to signing the form. If the subjects agreed to the conditions of the 
research, they signed and dated the consent form and returned it to the researcher. 

3. Interview Procedures 

The subjects were asked to answer in detail five questions based on their 
experiences or observations during the selected timeframe at the PEO or the DHMSM. 
Interviews lasted 30 minutes or less and were conducted either face-to-face, over the 
telephone, or via email. 

4. Privacy 

The Institutional Review Board (IRB) at the Naval Postgraduate School (NPS) 
required all subjects to be de-identified after the interview to protect their privacy. During 
the writing process, the research subjects are referenced with pseudonyms according to 
the positions they held during the selected timeframe. Examples include program 
executive 1, program manager 1, and manager 1. 

5. Interview Questions 

The interview questions were derived from the research questions: 

1. Were you involved in supporting any previous DOD EHR related 
programs? What do you believe were some of the key reasons for the 
failure of these past EHR programs? 

2. What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM program? 

3. What major acquisition strategies were implemented to overcome these 

obstacles and risks? 

4. How would you describe the managerial strategies of PEO DHMS and the 
DHMSM PMO, and how did they contribute to the success of the 
DHMSM program? 
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5. Based on your experience as a member of the PEO DHMS team, what are 
the key lessons learned and advice you can share that can benefit a DOD 
PM of other major software acquisition programs? 


F. RESEARCH BENEFITS 

The examination of the current PEO and DHMSM program provides a unique 
academic opportunity for learning how organizational behavior affects large DOD 
acquisition purchases. In addition, the lessons learned from a successful program can be 
used to streamline other major DOD acquisition projects and help support other federal 
agencies. When the DOD has the opportunity to implement lessons learned, it can be a 
better steward of the American taxpayers’ money, provide better services for military 
members and their dependents, and most important, potentially save lives. 

G. ORGANIZATION OF RESEARCH 

Chapter II provides a history of the electronic health record in the United States, a 
review of the DOD’s previous EHR acquisitions, as well as literature documenting 
implementation barriers and lesson learned. Chapter III presents the data from the 
research subjects’ interviews, and Chapter IV provides an analysis of the findings. 
Finally, Chapter V presents the researcher’s conclusions and suggests future research 
opportunities. 
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II. LITERATURE REVIEW 


This chapter includes a brief history of the electronic health record (EHR) and 
recent legislative developments. In addition, it discusses the history of the DOD EHR, as 
well as acquisitions and implementation failures. Moreover, it details the failed attempt of 
the DOD and VA to jointly build an EHR. This chapter also illuminates the challenges of 
other EHR implementation from around the world. Finally, the organizational and 
reporting structures of the Program Executive Office (PEO) are discussed. The literature 
review serves as a basis to understand what led to obstacles for the PEO and DHMSM. 

A. BACKGROUND 

In April 2004, President George W. Bush announced his goal that every American 
have an electronic health record by 2014 (White House, n.d.). He urged coordination 
between public and private sectors to accelerate adoption of health information 
technology. In 2009, President Barack Obama signed the Health Information Technology 
for Economic and Clinical Health Act, also known as the “HITECH Act,” with the goals 
of advancing the meaningful use of health information technology (HIT), improving the 
quality of care, and supporting the further development of EHRs. In addition to 
meaningful use, the act incentivized organizations that took on the risk of building an 
EHR (Centers for Disease Control and Prevention, 2016). According to Tripathi (2012), 
the HITECH Act helped create commonality in basic EHR functions, which propelled 
industry to drive the health information infrastructure. 

1. History of Electronic Health Record 

According to Ayatollahi, Mirani, and Haghani (2014), “EHR system is an 
information system that helps collect the health information of individuals from birth to 
death and provide real-time patient health records to support clinicians in decision 
making.” The main goal of implementing an EHR is to improve the quality of care by 
reducing medical errors by removing paper medical records and by increasing 
communication, data collection, and information sharing among health-care providers 
(Ayatollahi, Mirani & Haghani, 2014). 
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Prior to the 1960s, medical documentation was primarily paper based. During the 
1960s, electronic systems began to transform paper-based medical records into EHRs. 
The first generation held basic information such as the patient’s name, address, date of 
birth, social security number, and insurance information in a computer terminal (Egyhazy 
& Mukherji, 2004). Egyhazy and Mukherji (2004) explained that these systems were 
dormant and not user friendly. The information was not interoperable with other systems. 
Another difficulty came in that, in order to access information, the user had to access the 
computer in a fixed location. Advances in networking infrastructure that made it possible 
to access a patient’s medical information from other computer platforms came in the 
1970s and 1980s (Egyhazy & Mukherji, 2004). This led to the development of the DOD’s 
first EHR system, the Composite Health Care System (CHCS) in 1985. 

2. Military Electronic Health Record 

An EHR can play a vital role in the management of care received by members and 
their dependents over long periods of time. Within the military, a member’s health record 
is his or her living document from first entry into military services to veteran status. The 
electronic record must be available to health-care providers from multiple military 
treatment facilities in the continental United States (CONUS), outside continental United 
States (OCONUS or overseas), and in operational environments (Government 
Accountability Office [GAO], 2014). The large scale and required interoperability creates 
a challenging objective for DOD and increases risk for health-care operations. 

According to the Military Health System (2016a.), the DOD’s “EHR is used by 
more than 100,000 medical clinicians at military hospitals and clinics worldwide.” This 
EHR is used by the military’s medical community to capture, manage, and share data on 
care provided in garrison and on the battlefield (Military Health System [MHS], 2016a). 
In today’s military environment, there are two primary EHR systems in use: the 
Composite Health Care System (CHCS) and the Armed Forces Health Longitudinal 
Technology Application (AHLTA). Both systems are enterprise-wide information- 
management systems that provide secure access to health care records of active duty 
service members, retirees, and beneficiaries (MHS, 2016a). Both systems are explained 
further in subsequent sections. 


8 



The U.S. Department of Veterans Affairs (VA) uses an internally developed EHR, 
commonly known as the “best-of-breed” system. The program is called the Veterans 
Information Systems and Technology Architecture (VISTA). It provides an integrated 
inpatient, outpatient, and administrative functions within the EHR. VISTA supports over 
1200 healthcare sites, and it is one of the largest networks in the United States. It 
provides care to over eight million veterans, 163 hospitals, and 800 clinics through the 
United States (Department of Veterans Affairs [VA], n.d.). 

B. HISTORY OF DOD EHR 

This section goes into a brief history of the previous DOD EHR acquisitions. 

1. Composite Health Care System 

In May 1985, the DOD tasked the assistant secretary of defense for health affairs 
to acquire, implement, and operate a health care computer system. The total estimated 
cost was between $800 million and $1.1 billion, and the plan called to deploy CHCS to 
167 military hospitals and 577 clinics between 1987 and 1995 (GAO, 1986). 

During the acquisition of CHCS, numerous errors caused delays in system 
development contracting. First, the DOD miscommunicated with some vendors and 
allowed for unacceptable proposals to continue with contract actions when they should 
have been removed early in process, which increased costs and delayed the schedule. 
Second, the DOD’s disorganization with the industry proposals caused many proposals to 
go unreviewed (GAO, 1987). Third, the DOD failed to document previous discussions 
with industry that led to prior contract agreements. Finally, the DOD did not consider all 
cost factors at the beginning of the project; therefore, costs increased by $7 million when 
new proposals were added following discussions with industry (GAO, 1987). Overall, the 
CHCS system achieved what it was developed to do: provide electronic storage for 
service members’ health information. 

2. Composite Health Care System II 

In 1997, in an effort to improve the CHCS program, the Office of the Assistant 
Secretary of Defense for Health Affairs (ASD [HA]) directed the acquisition of a new 
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system to replace the legacy system. The new system was named the Composite Health 
Care System II, later known as the Armed Forces Health Longitudinal Technology 
Application (AHLTA). AHLTA was developed to address the limitations of the CHCS 
system. According to Egyhazy and Mukherji (2004), a major limitation was the program 
language used to develop the CHCS system. It was created with Massachusetts General 
Hospital Utility Multi-Programming System (MUMPS) programming language that did 
not allow for other modem programming languages to edit the code. Second, due to its 
unique programming language, the DOD lacked the ability to contract for commercial 
venders to train, maintain and evolve the system. 

The AHLTA system was developed differently from CHCS. The plan was to 
create an architecture with highly complex automation of systems, so AHLTA would 
comprise many other independent systems (Egyhazy and Mukherji, 2004). The 
acquisition timeframe was expected to last 18 years from fiscal year (FY) 1997 to 
FY2014 (GAO, 2002). According to a report by the DOD’s Inspector General Report 
(DODIG, 2006), AHLTA’s “full operational capability had been delayed by four years 
because of performance issues.” In addition, the dental module had to be redesigned to 
meet clinical needs. According to DODIG (2006), the system’s life cycle cost increased 
$1 billion (from $4,023 to $5,019 billion), and the program was extended for another 
three years. In 2012, the DOD cancelled all remaining updates to AHLTA. 

3. EHR Way Ahead 

According to the GAO, the DOD had obligated approximately $2 billion over 13 
years to AHLTA, the system that had not fully met its mission goals (GAO, 2002). The 
report mentioned that, even though the DOD had delivered many outpatient modules, many 
other capabilities originally planned for deliver had been scaled back. In addition, users 
frequently complained about AHLTA’s significant performance issues. To address these 
issues, the DOD recommended purchasing a new commercial system named “EHR Way 
Ahead.” Subsequently, the DOD budgeted $302 million for the EHR Way Ahead initiative 
(GAO, 2010). The DOD’s goal for this system was to be the “department’s comprehensive, 
real-time health record for service members and their families” (GAO, 2014). 
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4. Integrated Electronic Health Record 

During the same time the DOD was developing its plans for the EHR Way Ahead 
program, the VA also attempted to modernize its VISTA electronic health system. The 
VA had spent roughly $600 mi llion from 2001 to 2007 on projects to modernize VISTA; 
however, the department had estimated that the new cost would be closer to $11 billion 
with a completion date of 2018. The VA terminated the VISTA modernization effort in 
2010. In 2010, both DOD and VA secretaries signed an executive agreement to work 
together, and, in 2011, a new charter was signed to start the Integrated Electronic Health 
Record (iEHR) effort. This combined effort was expected to enable the VA and DOD to 
align resources and investments with common business needs that would reduce cost, 
increase efficiencies, and create an interoperable health information system (GAO, 2014). 

After years of developing iEHR, the DOD and VA announced the decision to 
abandon the effort in February 2013. This decision resulted from concerns over the 
program facing challenging deadlines, high costs, and an extended timeline to deliver 
capabilities. Based on these assessments, both the “DOD and VA would rely on separate 
systems to achieve an interoperable EHR” (GAO, 2014). Possible reasons for the failures 
of this program are discussed in Chapters III and IV, in which program managers share 
opinions on why this program failed. 

C. BARRIERS TO IMPLEMENTING AN ELECTRONIC HEALTH 

RECORD 

Several studies have shown the complexity in adopting and implementing an EHR 
system despite the potential benefits. This literature review suggests numerous 
organizations that have attempted EHR implementation have encountered road blocks 
during the process. These studies were conducted in the United States and other countries 
including China, Australia, Iran, and the United Kingdom. 

Research by Standing and Cripps (2015) suggested that EHR implementation 
downfalls are usually a result of complexities in the project scope, resources, decision¬ 
making authority, level of accountability, expectation by stakeholders, determination of 
staff, needs of the clients, and/or level of resistance. In a different study, researchers 
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Cucciniello, Lapsley, Nasi, and Pagliari (2015) documented managerial reasons that 
hinder EHR implementation. The researchers interviewed numerous clinicians and 
documented that failed relationships among stakeholders, patients, and clinicians can lead 
to potential barriers. In addition, the adoption of an EHR that affects the organization’s 
structure, culture, work processes, and communication channels will likely face 
resistance (Cucciniello et al., 2015). In a study of two EHR implementation projects in 
China, Gao, Xu, Sorwar, and Croll (2013) identified that a lack of governance, 
interoperability, legislation, and skilled professionals all contributed to challenges. The 
researchers agreed that a strong managerial strategy is critical for a successful project. 

One of the insights from Gao et al. (2013) and Cucciniello et al. (2015) was that 
people and their relationships are vital to a successful EHR implementation strategy. 
According to Standing and Cripp, the critical success factors that led a successful short¬ 
term EHR implementation rely on system focus rather than strategic focus. In other 
words, “a passionate and highly committed team can also make a big difference” in the 
success of the project (2015, p. 83). 

D. PEO DEFENSE HEALTH MANAGEMENT SYSTEM: NEW PROGRAM 

OFFICE 

This section covers the development of the PEO and DHMSM program offices 
and its reporting chain. 

1. Office of the Under Secretary of Defense for Acquisition, Technology 
and Logistics 

The Office of the Under Secretary of Defense for Acquisition, Technology and 
Logistics (OUSD [AT&L]) within the DOD oversees acquisitions, research and 
development, advanced technology, and logistics. Figure 1 provides a representation of the 
undersecretaries to the secretary of defense. Overall, the USD (AT&L) supervises all DOD 
acquisition projects and establishes policies for the procurement of goods and services, 
research and development, testing and contract administration (OUSD [AT&L], n. d.). 
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Figure 1. Organization Chart for the Office of the Secretary of 

Defense Principal Staff Assistants. Adapted from Deputy Chief 
Management Officer (2013). 


2. New Program Executive and Program Management Office 

With the shift in priorities after the iEHR project, both the DOD and VA 
announced they would pursue separate paths to obtain and modernize their respective 
EHRs. This led to the creation of the Program Executive Office (PEO)’s Defense Health 
Management Systems (DHMS). The PEO manages four joint programs under its 
portfolio. The PEO mission is to efficiently improve healthcare for active duty military 
members, veterans, and beneficiaries (Miller, 2015). One of its main missions is to 
modernize the electronic health record for the military health system (MHS). In June 
2013, the PEO established the Defense Healthcare Management System Modernization 
(DHMSM) program office. The DHMSM priorities have been to select and award a 
contract for a state-of-the-art EHR that will replace the legacy CHCS and AHLTA 
systems. The organizational chart for the PEO’s DHMS is provided in Figure 2. 



Figure 2. The PEO’s DHMS Organizational Chart. Adapted from MHS (2016c). 
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3. Defense Healthcare Management System Modernization 

With the establishment of the DHMSM, the program office has a clear mission to 
acquire, integrate, test, and deploy a modernized EHR system. In addition, the program 
will unify and increase accessibility of evidenced based healthcare delivery and decision 
making to the DOD MHS. The DHMSM’s organizational chart is shown in Figure 3. 
DHMSM has one program manager and a deputy program manager. In addition, it has 
aligned critical departments (assistant program managers) directly under the program 
manager. 



Figure 3. The DHMSM Organization Structure. Adapted from MHS (2014). 

4. Reporting Chain 

In 1986, the position of program executive officer was established; he or she will 
lead the PEO program office. A PEO is typically an executive office that oversees 
multiple program offices within the unique service component. Program offices have a 
strict reporting requirement to the PEO. Then, the PEO reports to the service component 
acquisition executives, as depicted in Figure 4. According to the Defense Acquisition 
University (2013b), this structure provides a clear line of authority for any high-cost and 
high-interest programs commonly referred to as Acquisition Category (ACAT) III, which 
referred to projects valued less than $185 million, ACAT II ($835 million or more), or 
ACAT I (projects of $2.79 billion or more). 
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In 2013, as the planning for the creation of the PEO program office was 
underway, the reporting chain to USD (AT&L) was modified. PEO no longer reports to a 
service component acquisition executive; the director of the program now reports directly 
to USD (AT&L), as shown in Figure 5. No other PEO program in the DOD or component 
services has this waver approved to directly report to the USD (AT&L). 



Under Secretary of Defense 
(AT&L) 


Assistant Secretary’ of 
Equivalent 


General Officers SES 
Civilian 


COL/CAPT/Civilian 

Equivalent 


Figure 4. Typical Reporting Chain for Acquisition Programs. Adapted from 

Defense Acquisition University (2013a). 
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Figure 5. Reporting Chain of Command for PEO to USD (AT&L). 

Adapted from MHS (2016b). 


5. New Contract Award 

In July 2015, the PEO and the DHMSM program office awarded a $4.1 billion 
contract to purchase a commercial-off-the-shelf (COTS) EHR from industry. The PEO 
streamlined the acquisition process to select and procure a state-of-the-art EHR in record 
time with no protests driving down the expected life cycle cost from $13 billion to $8.3 
billion. This thesis analyzes the acquisition strategy and management oversight used by 
the PEO to achieve results that surpassed all previous DOD acquisition attempts for an 
EHR (Mazmanian, 2015). 
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III. FINDINGS 


The findings presented in this chapter are the result of data collected by the 
researcher over four weeks in May and June 2016. The data represented in this chapter 
are the researcher’s summary of the subject interviews and an analysis of commonalities 
and differences among the responses. 

A. ORGANIZATION OF FINDINGS 

At the conclusion of each interview, the student researcher transcribed the 
subject’s words into an electronic format, which was used to visually organize 
commonalities and differences in the responses. The student researcher has grouped 
responses in this chapter based on the individual’s managerial category: executive, 
program manager (PM), or manager. The number of subjects by position in the hierarchy 
is tallied in Table 1. Hereafter, descriptive pseudonyms are used to represent each 
subject’s position. 


Table 1. Subject Tally by Position 


Position 

Number of Subjects 

Executive 

2 

Program Manager 

3 

Manager 

4 


Having three levels of the PEO/DHMSM hierarchy is important for determining 
successful management and acquisition strategies. Each level brings a different view of 
what is important or what challenges the program faced. Identifying commonalities and 
differences among levels of management should help reveal a set of best practices for 
future acquisitions of this size. 
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B. DATA COLLECTION 


In this section, the commonalities are identified among the subjects’ responses 
during the interviews. Responses are compiled into tables, and the frequency of answers 
are tallied according to the individuals’ positions. 

1. Question la: Were You Involved in Supporting Any Previous DOD 
EHR-Related Programs? 

Question 1 had two components. The first question asked whether the member 
was involved in any of the DOD’s previous electronic health record (EHR) system 
acquisitions or implementations. Table 2 shows the response breakdown by member and 
level. 


Table 2. Question 1, Part A—Were You Involved in Supporting 
Any Previous DOD EHR-Related Programs? 



Executive 

PM 

Manager 

TOTAL 

Yes 

1 

2 

2 

5 

No 

1 

1 

2 

4 

TOTAL 

2 

3 

4 



2. Question lb: What Do You Believe Were Some of the Key Reasons for 
the Failure of These Past EHR Programs? 

Part two of Question 1 asked the subjects to elaborate on their opinions and/or 
experiences surrounding the DOD’s failed EHR projects. The researcher recorded the 
responses from subjects who were involved in at least one previous DOD EHR project (see 
Table 3) and those who were not involved but gave their opinions anyway (see Table 4). 
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Table 3. Question 1, Part B—What Do You Believe Were Some of the Key Reasons for 
the Failure of These Past EHR Programs? “Yes” Respondents 


Executive 

Poor collaboration across organizations 

PM 

Attempt to develop or modify a product 


Unrealistic project scope 


Problematic requirements 

Manager 

Different internal processes 


Lack of commitment among both agencies (DOD and VA) 


Too many stakeholders, unwillingness to commit on a way forward 


Lack of single authoritative source 


Long lead times 


Multiple governance layers 


Lack of standard framework for data sharing 


Differences in organizational functions 


Table 4. Question 1, Part B—What Do You Believe Were Some of the Key Reasons 
for the Failure of These Past EHR Programs? “No” Respondents 


Executive 

Lack of stakeholder buy-in from users 


Lack of understanding of requirements 


Differences in organizational cultures 

PM 

Lack of acquisition professionals to purchase product 


Lack of buy-in from stakeholders 


Unsolidified program milestones 

Manager 

Lack of commitment 


Program focus on home-grown solutions 


3. Question 2: What Were the Key Obstacles and Risks Faced by the 

PEO DHMS and the DHMSM PMO at the Beginning of the DHMSM 
Program? 

Question 2 asked the subjects to identify any possible obstacles and/or risks that 
hindered the success of the program office. The researcher attempted to highlight the 
commonalities and differences in the subjects’ responses (see Table 5). 
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Table 5. Question 2—What Were the Key Obstacles and Risks Faced by the PEO DHMS 
and the DHMSM PMO at the Beginning of the DHMSM Program? 


Executive 

Lack of physical organization 


Lack of right people for the job 


Conflicting definitions of success or acquisition strategy 


Communication with U.S. Congress (politics) 


Communications with industry 


Differences in system requirements 


External politics from past EHR failures 

PM 

Bad press from U.S. Congress 


Lack of governance internal to the program 


Lack of requirements 


Establishment of program office 


Lack of funding 


Confusion over management structure 


Previous EHR failures 


Top-heavy bureaucracy 


Service conflict (Navy, Army, USMC, Air Force) 


Local command governance 


Unrealistic requirements 

Manager 

Lack of formal requirements 


Lack of empowered central leadership 


Aggressive schedule constraints 


Collaboration across multiple organizations 


Negative perceptions from previous EHR failure 


Lack of staffing plans 


Lack of credible cost estimates 


Inability to secure funding 


Lack of formal requirements 
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4. Question 3: What Major Acquisition Strategies Were Implemented to 
Overcome These Obstacles and Risks? 

Question 3 is a continuation of the previous question. It asked the subjects to 
identify actual strategies used that positively affected the program office and led to the 
successful award of the new EHR. The subjects’ responses are documented in Table 6. 
Question 3 leads to a macro view focused on the external factors that affected the two 
program offices. 


Table 6. Question 3—What Major Acquisition Strategies Were Implemented to 

Overcome These Obstacles and Risks? 


Executive 

Used competition effectively 


Engaged industry early and often 


Ensured requirements were right 


Developed a tailored and flexible contract strategy 


Used multiple industry days to communicate to industry 


Released requests for information (RFIs) before finalization 

PM 

Tailored acquisition documents to streamline approval process 


Reported to USD (AT&L) 


Leveraged lessons learned from commercial industry 


Created no artificial requirements 


Focused on commonalities of a uniform health care system 


Allowed for an aggressive strategy 


Emphasized streamlined processes 

Manager 

Rebuilt broken relationships 


Overcame staffing obstacles by outsourcing from DHA and Space and 

Naval Warfare Systems Command (SPAWAR) 


Reported directly to USD (AT&L) 


Challenged the status quo 


Tailored the acquisition plan 


Developed requirements through draft RFPs 


Partnered with industry 
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5. Question 4: How Would You Describe the Managerial Strategies of 
the PEO DHMS and the DHMSM PMO, and How Did They 
Contribute to the Success of the DHMSM Program? 

Question 4 asked the subjects to identify specific management strategies used in 
either program office. In contrast to question 3, this question offers a micro view focused 
on the management strategy within the two groups. The subjects’ responses are 
documented in Table 7. 


Table 7. Question 4—How Would You Describe the Managerial Strategies of the 
PEO DHMS and the DHMSM PMO, and How Did They Contribute to the 
Success of the DHMSM Program? 


Executive 

Communicated with all levels 


Used a multi-dispensary team 


Brought in acquisition leadership 


Allowed industry more time to plan before final RFP 

PM 

Flattened organization; reduced layers and bureaucracy 


Developed meetings with a purpose 


Created relationships with requirements community 


Developed static requirements 


Leveraged previous market research 


Used complete transparency 


Used aggressive scheduling with milestones 


Published acquisition plans 


Met with industry often 


Adopted a “hands-on” leadership approach 


Forced people to make decisions 


Engaged the functional community directly 

Manager 

Released information early 


Identified the right skill sets and staff needed 


Ensured the right community was part of the functional group 


Employed leadership with COTS acquisition experience 
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Brought in highly motivated staff 


Used a tailored acquisition process 


Established authority to proceed 


6. Question 5: Based on Your Experience as a Member of the PEO 

DHMS Team, What Are the Key Lessons Learned and Advice You 
Can Share That Can Benefit DOD PM of Other Major Software 
Acquisition Programs? 

Question 5 asked the subjects’ opinions on future program-office strategies from 
which other large acquisition programs can learn. The subjects’ responses are 
documented in Table 8. 


Table 8. Question 5—Based on Your Experience as a Member of the PEO DHMS 
Team, What Are the Key Lessons Learned and Advice You Can Share That 
Can Benefit DOD PM of Other Major Software Acquisition Programs? 


Executive 

Under-promise and over-deliver 


Communicate 


Define success/acquisition strategy 


Adopt the enterprise system 


Interact early with industry 


Employ change-management and training early 

PM 

Collect true requirements 


Develop a solid acquisition strategy 


Adopt a COTS product with commercial capability 


Allow enough space to leverage industry 


Be descriptive in requirements 


Have good decisive and leadership 


Stay within organizational functions 


Involve functional (medical) community 


Force functional community to define requirements 


Select leaders who are not tied to the medical community 


Set-up a healthcare systems and material command (SYSCOM) 
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Solidify the true requirements 


Leverage commonality—don’t highlight differences 

Manager 

Take risks 


Streamline the acquisition process 


Focus on teamwork; communicate 


Establish realistic goals and timelines 


Create positive working environments that motivate employees 


Have a clear understanding of requirements 


Initiate early interactions with industry 


Force organizations to formalize requirements 


Empower a central leader to make decisions for the health-care 

enterprise 


Remove customers from the acquisition decision; leave business 

decisions to acquisition people 


C. SUMMARY 

Responses from Question 1 highlight the cultural differences between the DOD 
and the VA, which led to the iEHR failure to mature to a program of record. Responses 
from Question 2 expanded the uncertainties in standing up a program office. The 
concerns expressed were related to external pressures placed on the program office. On 
the other hand, in answers to Questions 3 and 4, it appears that the program office moved 
from a state of uncertainty to a determined, mission-focused organization. The 
management strategy and requirement gathering were important during this phase. 
Question 5 yielded some unique suggestions for improving future acquisitions but 
reiterated the experiences from Questions 3 and 4. Chapter IV further analyzes the data 
presented in Chapter III. 
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IV. ANALYSIS AND DISCUSSION 


A. ANALYSIS 

The success of the Defense Healthcare Management System Modernization 
(DHMSM) and Program Executive Office (PEO) is broken down into five strategic 
improvements. These improvements directly led to the successful planning and 
acquisition of the DOD’s EHR system in July 2015. These five improvements include the 
following: true requirements, the program governance structure, hands-on management, 
external human resources, and the separation between acquisition and medical 
communities. In this section, we use the subjects’ interview responses as discussion 
points to the five strategic improvements. These strategies are unique to the PEO and 
DHMSM programs. 

1. Requirement Gathering 

In software engineering, requirement gathering deals with establishing the needs 
of the stakeholders that the software fulfills. During the acquisition of the new EHR, the 
acquisition team was in charge of purchasing COTS software, but the medical 
community was not part of this team. It maintained a separate community, known as a 
“functional community.” The functional community provided the acquisition team with 
information on medical necessities for the new software to perform. 

a. True Requirements 

Since the new EHR was a COTS product, it was critical that the correct 
requirements were collected prior to the decision to purchase the software. Program 
manager 2, who observed the DHMSM program, expressed the need for the medical 
community to clearly define the EHR’s requirements: “They really need to be forced to 
define the requirements. The tighter you get, the better you’ll be able to buy it for them. 
The better the requirements, the better the acquisition will be” (personal communication, 
May 27, 2016). Words from program manager 2 support the finding that a partnership 
between the acquisition and functional communities were critical for success. The 
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functional community performs that first step by carefully selecting the specific tasks 
clinical professionals need from the EHR to provide medical care for patients. The 
acquisition community takes those specific tasks and attempts to purchase a product that 
meets the needs of the functional community. 

Specific requirements are necessary to purchase a software product; however, 
sometimes having too many requirements causes delays or excess costs when the 
software is purchased. This is the importance of having true requirements provided by 
the functional community. Medical professionals have the tendency to add requirements 
based on special medical processes, which add to the complexity of an EHR. Program 
manager 1 suggested, “We should not create artificial requirements based on the 
environment we operate in. We should make sure that the requirements are solidified as 
true requirements and differences in a way ... solutions are presented” (personal 
communication, June 10, 2016). Program manager 1 echoes the findings from the survey 
that the medical community needs to provide the acquisition community a true set of 
requirements. In doing so, the acquisition community better negotiated with industry to 
purchase a cost-efficient product. 

Medical fields are dynamic; they evolve continually. The nature of medicine 
creates a challenge when a group of clinical professionals are tasked with developing 
requirements for an EHR. The requirements have to be flexible to allow for changes in 
medicine and the specifics of medical practice. However, the acquisition community 
needs fixed requirements. Otherwise, flexibility or indecision results in lengthy, costly 
evaluation cycles. Program manager 1 commented on what acquisition teams do to work 
around this problem: 

Acquisitions are static; the environments that they intend to support are 
dynamic. So you have to build contract vehicles that are flexible and 
evolving around it. We know that the requirements will not be static at the 
time we buy the product. We have to present a static requirement with the 
framework of the contract with the ability to be flexible as the requirement 
changes. That’s a fine line that not everyone understands. (Personal 
communication, June 10, 2016) 
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Even though the acquisition process and the requirements need to be static; the contract 
can be written to allow flexibility. In the new EHR contract, additional line items allowed 
for future modifications. 

b. Partnership with Industry 

During the development of the EHR’s capabilities, the functional community 
continued to disagree over what true requirements should be used. The program office 
successfully used industry to help fill in the gaps for the medical community. Manager 1, 
who was involved with the EHR acquisition selection process, discussed using industry 
to fill the requirements gap to support the functional community: 

I mentioned that they [medical community] did not have well-articulated 
formal requirements, but what the program office did was they partnered 
with industry and utilized an iterative draft RFP process to help articulate 
requirements that industry can answer. It was brilliant because the services 
couldn’t really say what they wanted, but they had an idea [of] what they 
wanted. (Personal communication. May 26, 2016) 

The DHMSM program office frequently used industry as a partner to build its 
requirements. What DHMSM did was unique in that it released to industry a request for 
information (RFI) early during the planning process. Even though the RFI was 
incomplete, meaning that the program office did not fully know its final requirements, the 
office released it to industry for feedback. The DHMSM program office continued to 
release many other RFIs during the planning phase. This led to the development of a rock 
solid request for proposal (RFP) that industry could fully support. Program executive 2 
said this regarding the RFI process: 

In the past, we in government waited until we had the RFP completed 
before forwarding to industry. This time, we took our first cut, which still 
needed work but we shared it with industry. We got their feedback and 
continued to get internal feedback to make the RFP better. I think that’s a 
big lesson learned: even though it’s not ready for prime time, share that 
RFP early. (Personal communication, May 16, 2016) 

Sharing the RFP early and often allowed the program office to increase visibility 
with industry and help build the right requirements for the EHR from commercial lessons 
learned. Both executives and managers expressed that developing requirements to 
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populate an RFI was an obstacle, as shown in Table 5. However, as one program 
executive mentioned in Table 7, the use of industry early to develop the requirements was 
noted as a best practice that helped the government receive the best bids at very good 
prices. 


2. Governance 

As discussed in Chapter I, no other program executive office has been governed 
like the PEO DHMS. The PEO was given a direct-reporting status to the USD (AT&L), 
as represented in Chapter I, Figure 5. Normally, a PEO falls under a major command to 
govern its actions. One such organization is the Defense Health Agency (DHA), which 
reports directly to the secretary of defense. Because the program originated in Congress, 
it was high profile, and Congress controlled the cost. Therefore, selecting the right 
leadership and subordinate was key to governance. Manager 2, a seasoned PEO 
administrator, described how USD (AT&L), the head of PEO DHMS, was crucial to 
success: 

The years of acquisition experience enabled leadership to identify the 
process inherently required for a successful COTS acquisition. They 
identified the right skill sets and staff needed to meet the challenges of 
developing an RFP that clearly addressed and identified the requirements 
... for a vendor to provide a COTS solution. Additionally, through their 
leadership, they ensured the right communities were part of the evaluation 
process to ensure the functional requirements would be met. In addition to 
their leadership, folks were highly motivated to support the mission of 
obtaining a cradle-to-grave EHR for our service members, families, and 
beneficiaries. (Personal communication, May 25, 2016) 

One smart decision leads to others. U.S. Congress mandated that the secretary of 
defense modernize the MHS’s electronic health record. Therefore, the DOD created a 
structure for the PEO to align directly under the USD (AT&L). The PEO then created the 
DHMSM program office to directly report to it. Therefore, the PEO had only one 
reporting chain, and DHMSM had two reporting chains. It was unprecedented for an 
AC AT I program of its size to have minimal governing layers. PEO manager 2 
commented on the benefits of directly reporting to USD (AT&L): “Having an authority 
figure such as the secretary of defense or USD (AT&L) ... influences decisions from 
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services’ figures that keep your program moving. The petty bureaucratic fights can hold 
down progress” (personal communication, May 31, 2016). This hierarchy was vital as all 
services—Army, Navy, Marine Corps, and Air Force—report directly to the USD 
(AT&L) for their procurements. The PEO can force the services to work together and 
come to terms because the USD (AT&L) controls funding and procurement. This 
resolved any disagreement among the services and kept the project moving forward. 
Program manager 1 emphasized how this tailored acquisition process reduced 
bureaucracy: 

The managerial strategy was to flatten out the organization. There were 
too many layers. When you are in a compressed time line to deliver, you 
can’t have a lot of bureaucracy at lower levels where decisions are 
occurring, so USD (AT&L) fosters that. Access to leaders that make 
decisions is imperative. (Personal communication, June 10, 2016). 

This program manager noticed the difference when he had direct access to 
decisions to keep the program moving. As shown in Table 7, both program managers and 
managers mentioned that a streamlined reporting chain helped them obtain decisions 
from leadership sooner. Typically, the governance of the services and services’ medical 
departments can hinder the progress. However, having the direct support of the PEO and 
USD (AT&L) helped. 

3. Management Styles 

The management style of the leaders from the PEO and DHMSM was another 
important factor for the success during the new EHR acquisition. The leadership in both 
program offices was described by those surveyed as very aggressive for results. The PEO 
and DHMSM felt pressure to deliver a product by a specific date and cost, but one of the 
risks was the continued indecisiveness of the functional community in laying out 
requirements. However, program manager 2, who observed the process, mentioned how 
the aggressive management styles worked in favor of moving the project forward: 
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The PEO and DHMSM program managers were very hands-on people. 

They looked to engage the functional community directly and force people 
to make decisions and get them on board. They forced people to have 
discussions on where the fiction points were and to overcome them. You 
do that by transparency and talking to people. (Personal communication, 

May 27, 2016) 

In addition, the leadership at the PEO maintained a high tempo by holding weekly 
meetings on various deliverables from the DHMSM program office. This pace ensured 
that the program office was on target and delivered each milestone on time. Though some 
might view this style of management as micro-management, it worked. 

4. Human Resource Sourcing 

Another obstacle that the DHMSM program faced at the beginning of the 
acquisition process was the creation of a physical office. As the survey noted from the 
executive, program manager, and manager levels, one consistent concern was the 
establishment of a program office that would be responsible for purchasing the EHR 
system (see Table 5). The ability to find the right staffing for the positions at DHMSM 
proved challenging given the aggressive timeline for the PEO. Manager 2, who was 
involved with the PEO from the beginning of the program, explained the obstacles faced 
with the DHMSM program: 

A clear plan for staffing and manning the projects and programs was 
needed. Then obtaining the right talent, in a timely manner, to execute a 
plan that is filled with challenges ranging from long hiring times for 
civilians[.] [There were] service members’ rotation schedules and clearly 
obstacles in being able to accomplish the work. (Personal communication, 

May 18, 2016) 

This manager’s worries were legitimate because for the federal government, 
hiring from the civilian sector could take many months of advertisement, recruitment, and 
training to get new personnel on board. Given the specific knowledge and skills required 
for the job and the aggressive acquisition timeline, this would prove a nearly impossible 
task. These concerns are noted in Table 5, wherein both the executives and program 
managers showed concern about building a program office from scratch. Manager 2 
explained that the “staffing obstacles were overcome with the agreement between DHA 
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and SPAWAR—allowing the PEO DHMS to hire skilled staff from the working capital 
funds” (personal communication, May 18, 2016). Essentially, the PEO and DHMSM 
cleverly borrowed personnel from other program offices and staffed them locally. Even 
though government contractors could have covered the majority of resource gaps within 
both organizations, government expertise was critical for leading and holding action 
accountable, which ultimately led to the success of the EHR system. 

5. Maintaining Functional Community Responsibilities 

The separation between acquisition and medical community responsibilities was 
also critical. The DHMSM program was set up with acquisition managers leading and 
medical professionals providing guidance. This organizational structure was different 
from previous EHRs, wherein the medical community was closely involved in the 
acquisition process. The development of the altered organizational structure started with 
the USD (AT&L) selecting an acquisition professional to lead the PEO program. In turn, 
the PEO selected another professional with extensive acquisition-purchasing and COTS 
experience to lead the DHMSM program. Program manager 2 observed how separating 
the two communities affected the acquisition process positively: 

The leaders selected to lead the acquisition should not be directly tied to 
the community that they are trying to service. The program manager of 
DHMSM was not a medical doctor, and PEO was not a medical guy. I’m 
not a medical guy; I’m an acquisition professional. I have medical people 
here that can educate me and advise me on what we need the system to do. 

But I’m not tied to a particular community... [I] wanted to get the best for 
the government and at a good price. There’s no loyalty I feel to a 
community. Nothing in my past experience influences me in a certain 
direction. (Personal communication, May 27, 2016) 

Program manager 2 echoes the findings from the survey that medical 
professionals may present a conflict of interest in the acquisition process, as shown in 
Chapter III, Tables 5, 7, and 8. The goal of the procurement was to purchase a state-of- 
the-art EHR system at the best price for the government. The individuals best suited to 
negotiate for the government were those who did not have a conflicting interest in the 
procurement and understood how the acquisition process works. Medical professionals 
are not trained to handle large acquisitions, as program manager 1 explained: 
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Doctors and nurses are not trained to do acquisitions; their forte is to 
deliver care to needy patients. You can’t treat the acquisition community 
as if it’s not its own area of expertise. You must be an expert in acquisition 
and contracting to do multi-billion-dollar deals. (Personal communication, 

June 10, 2016) 

The program manager expressed that it took experience and acquired skills to 
conduct large-scale acquisitions properly. The program manager strongly suggested that 
the acquisition community is specially trained to manage these type of projects. There is 
a perception in the medical community that if the acquisition is a medical system they 
should have a lead in the project. Program manager 1 explained the role of the clinical 
and acquisition personnel: 

The clinical component is struggling in this area. They got to get an 
acquisition professional in their portfolios and then allow doctors to 
practice medicine, which they are allowed to do. The functional piece is 
the doctors’ piece. ... The acquisition piece, leave it to the acquisition 
professional. (Personal communication, June 10, 2016) 

The DHMSM program was configured to have two acquisition leaders manage it. 
The medical component under the chief medical information officer (CMIO) was an 
assistant program manager to the DHMSM program manager, as shown in Figure 3. The 
CMIO coordinated information gathering among the functional community to formulate 
the necessary requirements for the program office. This strategy worked for DHMSM. 

B. DISCUSSION 

In this section, we revisit and answer the research questions from Chapter I. The 
interview questions were derived from the main research questions. Data collected from 
Chapter III help guide the answers to the research questions. 

1. What Were the Key Reasons for the Failures of the Previous 
Enterprise EHR Programs within the DOD? 

From the survey, Question 2 presented the research subjects’ responses. The 
responses were based on experiences and/or insider knowledge (see Table 3 and 4). Some 
of the research cohorts had had numerous experiences with health IT and DOD 
acquisition projects; therefore, they could speak in detail about the subject. 
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Program executive 2 suggested that the previous DOD EHR systems were not all 
failures. The Composite Health Care System (CHCS) and Armed Forces Health 
Longitudinal Technology Application (AHLTA) programs did not achieve all they set out 
to do, but the systems did move the military health system toward a functional EHR 
(personal communication, May 16, 2016). However, the integrated electronic health 
record (iEHR), which was supposed to encourage the DOD and VA collaboration, failed 
to achieve its mission. This section discusses the opinions of why the iEHR failed and 
why VA and DOD decide to go separate directions. 

a. Differences in Organizational Culture 

Among those interviewed who direct exposure with the previous EHR, 
experienced had shared that both organizations had difficulty collaborating due to 
difference in internal process (see Table 3). One of the differences was the inability to 
decide on system requirements. During this time, the VA wanted to update its 
homegrown Vista solution while the DOD wanted to purchase a new COTS system. This 
created problematic requirements, which developed into an unrealistic project scope. In 
addition, there were many stakeholders from medical communities from both 
departments who added to the requirements. The iEHR program could not move forward 
due to the levels of bureaucracy, which created longer than normal lead times. 

Argumentatively, the iEHR program was destined to fail due to its size and 
complexity. The program office and its leadership were not assertive in maintaining 
control of the complex system; therefore, the program lost track of its goals. 

2. What Obstacles and Risks Did the Program Office Face When 
Implementing the New System? 

This section aligned with Question 2, where it asked the research subjects to 
express their experiences or opinions of what risks DHMSM faced prior to the 
acquisition and implementation of the new DOD EHR system (see Table 5) 
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a. 


Previous Failures 


Executives, program managers, and managers all echoed that the iEHR’s previous 
failure had a profound effect during the early stages of the program. The negative 
publicity received from the iEHR and other past EHR systems had many individuals 
doubting whether the DOD would be able to procure a new system. The negative 
publicity affected future funding and working relationships. According to manager 2, the 
new PEO head worked hard to reestablish trust within the DOD and its stakeholders. The 
PEO actions allowed the PEO and DHMSM program to get on its way (personal 
communication. May 25, 2016). 

b. Uncertainty 

Executives, program managers, and managers all exhibited concerns over the 
many uncertainties during the creation of the DHMSM program office. The majority of 
managers’ uncertainties were tied to standing up the program office within the aggressive 
timeline. Many managers noted that staffing, confusion over the management structure, 
and a lack of formal requirements were major concerns. The executives were overly 
concerned with the conflicting acquisition strategy and communication with industry, 
whereas the program managers were concerned about service interactions and the 
confusion over the management structure. 

3. What Acquisition and Managerial Strategies Directly Led to Success? 

This section used survey responses from Questions 3 and 4. Most certainly, the 
tailored acquisition that allowed the PEO to report directly to the USD (AT&L) was a 
major factor; however, there were other strategies that may have contributed to the 
success. 


a. Industry Communication 

Executives, program managers, and managers emphasized the importance of early 
communication with industry in helping with the requirements collection and 
development. The DHMSM program used requests for information (RFI) effectively to 
communicate with industry. A typical program office would spend excess amount of time 
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to ensure that the RFI would be contain complete and correct information before it is 
shared with industry. This method can be ineffective, as industry may return the request 
back to DOD as something impossible to do. Early communication means providing 
industry limited feedback (requirements) on what they need, then the commercial sector 
will be able to fill in the blanks on what it can provide to the federal government. The 
federal government then takes the suggestions by industry and continues to build on its 
requirements. Having industry tell the government what it can provide early is extremely 
effective because the government is trying to leverage a COTS product. The government 
should not develop requirements that industry cannot meet, and early communications 
helps reduce that risk by allowing the commercial sector do the market research for the 
DOD. 


4. What Can Current Program Managers of Major Software 
Acquisition Learn from the DHMSM Program? 

This section refers to Question 5 from the survey. There were numerous 
suggestions by the research subjects on possible lessons learned. The commonalities 
among the executives, program managers, and managers were the early interaction with 
industry and properly adopting the COTS product. The commonalities between program 
managers and mangers were decisive leadership, formalize requirements, streamline 
acquisition process, and separation between acquisition and functional communities. 

C. SUMMARY 

In summary, the analysis section provided five improvement strategies that were 
critical to the success of the DHMSM and PEO efforts to procure the DOD’s new EHR. 
The governance structure provided PEO top cover to manage the services’ expectations 
and future funds. Without the support from the USD (AT&L) for the PEO, the program 
could have fallen behind schedule. The collection of true requirements ensured that the 
DHMSM program purchased an EHR system that was state-of-the-art and cost effective. 
Sourcing from other government organizations helped support the PEO and DHMSM 
programs to ensure the right subject matter experts filled critical resource gaps quickly. 
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The separation between the acquisition and medical communities ensured that conflicting 
interests did not add to the bureaucracy and that personnel focused on project objects. 

The leaders’ management styles allowed all of these strategies to happen. The 
hands on leadership of the PEO and DHMSM programs ensured that the requirements, 
resources, and governance were established, functional, and operational. For major 
information technology (IT) acquisitions, strong, engaged, and aggressive leadership is 
needed with the addition to political top cover. 

This chapter revisited the research subjects’ responses from the survey. Failures 
of a program office generally stem from numerous factors, but the subjects’ experiences 
linked the failure of the iEHR to differences between the two departments’ (i.e., the DOD 
and VA) cultures and the inability for leadership to bring the requirements toward a 
common goal. The challenges the PEO and DHMSM largely overcame the history of past 
EHR failures and the negative perspectives of stakeholders. This resulted in many project 
uncertainties. Overall, the use of industry as a partner helped ensure the program office 
procured a state-of-the-art system at a very good price for the government. 
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V. CONCLUSION AND RECOMMENDATIONS 


This chapter summarizes the importance of this thesis and discusses the data used 
to conduct the study. In addition, the chapter contains conclusions drawn from the 
analysis and discussion in Chapter IV. Finally, the thesis ends with recommendations and 
suggestions for future studies. 

A. CONCLUSION 

The information presented in this research can be used as a lesson learned for 
future large DOD or federal acquisition programs. Moreover, the information collected is 
extremely valuable to the acquisition community supporting medical systems. Due to the 
history of the DOD’s EHR programs, the data collected in this research can serve as 
building blocks for improving future medical acquisition processes as well as all major 
acquisition programs. The ramifications of not implementing these best practices could 
result in repeating previous failures. 

Generally, failure in IT systems is not due to the technology itself; it is due to the 
inability of the users to train fully and implement the system into their daily work flows 
(Cripps & Standing, 2015). In program management, the successful acquisition of a 
system requires motivated leadership and knowledgeable staff members to navigate 
obstacles effectively and develop management strategies to succeed. In the case of the 
DOD’s EHR system, negative press and congressional mandates had already created an 
obstacle for the PEO even prior to the start of the program. Decisive leadership 
understood what it needed to make the process work effectively, and it did something that 
went against the grain of previous acquisition projects. Implementing a tailored 
acquisition process for the PEO and DHMSM programs reduced the amount of red tape 
for critical decisions. Lessons learned from the past ensured that leadership placed the 
skilled acquisition program-managers needed to lead the procurement and to keep the 
functional community engaged—while not slowing down the process. Instead of 
alienating the commercial sector, leadership used industry effectively as a partner. This 
was shown to be a best practice as the commercial sector gave its input to build the 
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DOD’s EHR, played a part in the process, and equally marketed products. When the 
contract award was announced, there was no protest from industry, and this is very 
unusual in a large dollar contract. The following are recommendations drawn from this 
research for future program offices. 

1. Create a Tailored Acquisitions Process 

Reduce the amount of red tape for program managers to get critical information 
approved by the stakeholders. The majority of programs suffer the additional burden of 
long waiting periods before a key decision can be made. Allow the program manager 
access to executives who are able to influence the decisions of multiple stakeholders. 
Streamlining the acquisition process should also include the managers’ access to the 
program managers for decisions as well. 

2. Separate Acquisition and Functional Communities 

Traditionally, medical professionals serve an important and influential role in the 
program management of a health information technology system. This could possibly 
create a conflict of interest wherein the medical professionals may unintentionally add 
requirements that are unnecessary for an enterprise system. Separating the roles—where 
acquisitions is in charge of procurement and the functional community advises—helps 
reduce the risk of adding requirements that are not needed. 

3. Create Partnerships with Industry 

Normally, the federal government waits until it has a complete proposal with all 
requirements before sharing any information with industry. With the DHMSM program, 
information was shared during the request-for-information stages before all requirements 
were gathered. According to executive 2, members of industry appreciate the early 
notification because it allows them to gather their teams and begin planning before the final 
request for proposal. Executive 2 reiterated that in many instances, industry flies blind as to 
what the government needs. Industry needs a lot of lead time to prepare for bidding on 
large programs. The federal government will benefit from the extra lead time for industry 
because it yields more competitive bids (personal communication, May 16, 2016). 
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4. Motivated Leadership 

For COTS procurement, a knowledgeable leadership in acquisition processes with 
hands-on program governance can help move the system purchase forward and keep it on 
track. Leadership needs to engage the functional community directly. The PEO head 
should engage the functional community at the highest levels with the support of the 
USD (AT&L), forcing services to collaborate and buy-in to the system. The DHMSM 
program manager should aggressively engage the functional communities’ committees to 
provide true requirements to the acquisition team in a timely manner. These leadership 
styles were able to bridge the two communities by ensuring that difficult decisions were 
made without continued friction, a critical piece to the success of both programs. The 
aggressive leadership displayed during PEO and DHMSM programs was successful in 
procuring a system; however, such a strategy may not be as effective for sustainment 
efforts. 

B. FUTURE STUDY 

This study provides an examination of the successful strategies the PEO and 
DHMSM used during the acquisition of the DOD’s new EHR. It serves to identify what 
entities may influence a successful acquisition program-management office. Referring to 
Table 8 in Chapter III, the survey responses may offer a guide to determine future 
studies. The following are recommended areas for future study. 

1. Examine the development of health systems and material commands. 

2. Examine the improvement for standardizing separating acquisition and 
functional activities for large program projects. 

3. Examine the impact of leveraging the use of the commercial industry early 
during the acquisition process. 

4. Examine implementing a streamlined reporting chain for major acquisition 
programs to reduce bureaucracy. 

5. Examine how organizations may codify and articulate their requirement 
collection to result in a successful system. 


39 



THIS PAGE INTENTIONALLY LEFT BLANK 


40 



APPENDIX A. PROGRAM INTERVIEWS—PROGRAM 

EXECUTIVES 


The following are excerpts from telephone interviews from the program 
executives. Personal identifiable information has been removed to protect the subjects’ 
privacy. 

Executive 1 

Interviewed on June 8, 2016 

Interview conducted via telephone 

Question 1) Were you involved in supporting any previous DOD EHR related 
programs? What do you believe were some of the key reasons for the failure of these past 
EHR programs? 

• No, have not been involved in previous DOD EHR programs. 

• Possible lack of understanding of requirements. 

• Possible lack of stakeholder buy-in from users. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM program? 

• Lacking physical organization then had to stand up organization. 

• Finding the right people for the job. 

• Defining what success looks like and pulling together the acquisition 
strategy. 

• Communications with congress and services. 

Question 3) What major acquisition strategies were implemented to overcome 
these obstacles and risks? 

• Effectively used competition and engaged industry early and often. 

• Industry understood requirement early. 

• Ensure the requirements are right and build on them. 

• Had DHA and services spearhead requirements development. 

• Developed contract strategy that was tailored and flexible. 
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Question 4) How would you describe the managerial strategies of PEO DHMS 
and the DHMSM PM, and how did they contribute to the success of the DHMSM 
program? 

• Communicate with people with what is happening. 

• Use a multi-dispensary team. Use people with all kinds of technical 
abilities. 

• Ensure an open process on [how] PEO is executing the program. Held 
monthly reviews and invited stakeholders. 

Question 5) Based on your experiences as a member of the PEO DHMS team, 
what are the key lessons learned and advice you can share that can benefit DOD PMO of 
other major software acquisition programs? 

• Must communicate. 

• Under promise and over deliver. 

• Must adopt the enterprise system. 

• Must have change management and training for user adoption. 


Executive 2 

Interviewed on May 16, 2016 

Interview conducted via telephone 


Question 1) Were you involved in supporting any previous DOD EHR related 
programs? What do you believe were some of the key reasons for the failure of these past 
EHR programs? 

• Yes, was involved with ALHTA, iEHR and MHS Genesis. 

• CHCS was not a failure, it moved MHS forward. ALHTA was also not a 
failure, it did not achieve everything is was set out to achieve, as several 
capability blocks were not delivered. However, it moved ambulatory care 
and documentation forward, [and] moved electronic health repository 
forward, first time any physician can see a health record at any part of the 
world. In that regard, it was a success but it did not achieve everything it 
set out to do. 

• iEHR was different. DOD and VA wanted the next generation electronic 
health record to be for both DOD and VA. VA and DOD had different 
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views on how the EHR should proceed because DOD looks at 
Commercial off-shelve (COTS) but VA had been a pioneer with electronic 
health record development but has fallen behind commercial EHR system 
capabilities. 

• Commercial capabilities have over taken Vista capabilities. DOD made a 
compromise to try to work with the VA on iEHR, instead of going with 
the best of suite choice, they decided to with VA and go with a best of 
breed approach with the iEHR. There were reports of cost overruns but 
there wasn’t really cost over runs b/c there wasn’t a contract for iEHR yet. 
The program was stopped before the best of breed occurred. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM program? 

• We were coming out of iEHR and there were still a lot of people that 
thought we should still continue to work with VA and use their VISTA 
system. But we believed that VISTA was then behind the market and 
would be costly to upgrade and maintain over time. DOD decided that 
VISTA wasn’t an option. That was an obstacle because there is alot of 
politics involved. 

Question 3) What major acquisition strategies were implemented to overcome 
these obstacles and risks? 

• Multiple RFIs in areas we had questions on certain aspects of the EHR 
market places. So we did targeted RFIs and draft RFPs and the industry 
days. 

• From the 30 years of experience, for the large scale systems and middle 
scale systems, this is the best interaction with industry I’ve seen, to get the 
best bids at a very good price to meet the needs. 

• In the past, we in government waited until we had the RFP completed 
before forwarding to industry. This time, we took our first cut which still 
needed work but we shared it with industry. We got their feedback and 
continued to get internal feedback to make the RFP better. I think that’s a 
big lesson learned, even though it’s not ready for prime time-share that 
RFP early. 

Question 4) How would you describe the managerial strategies of PEO DHMS 
and the DHMSM PMO, and how did they contribute to the success of the DHMSM 
program? 

• The strategy that [PEO and DHMSM heads] brought with them combined 
with knowledge of the folks that lived with the iEHR and ALHTA/CHCS 
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enabled us to put together an excellent acquisition. I think that success 
when you combine the history lessons learned with the new thinking on 
how to interact with industry really led to the success. 

• Industry appreciates the draft RFP because it allows them to get ready 
their teams and begin planning before the final RFP. 

Question 5) Based on your experiences as a member of the PEO DHMS team, 
what are the key lessons learned and advice you can share that can benefit DOD PM of 
other major software acquisition programs? 

• Inaction with industry. A lot of times, industry is flying blind on what the 
government really needs. Industry needs a lot of lead time to prepare for 
any large program, to get their teaming right, and to do a competitive bid. 
There’s a lot of work that needs to be done on the industry side. Putting 
out RFPs, [and] RFIs, not only allows industry the ability to provide input 
to the whole process but also allow them to do work to get their team 
together [and] to prepare to bid on the RFP when it comes out. The more 
time you give industry, the better off they will be to offer a completive and 
good bid and it totally benefits the government when they can do that even 
though you are not ready for prime time with your RFP yet. So being able 
to do that early communication is tremendous for these companies because 
they are literally flying blind when stuff comes out from the gov’t as a 
surprise. 
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APPENDIX B. PROGRAM INTERVIEWS—PROGRAM 

MANAGERS 

The following are excerpts from in-person interviews from the program 
managers. Personal identifiable information has been removed to protect the subjects’ 
privacy. 

Program Manager 1 
June 10, 2016 

Interview conducted via face-to-face 

Question 1) Were you involved in supporting any previous DOD EHR related 
programs? What do you believe were some of the key reasons for the failure of these past 
EHR programs? 

• No, was not part of any previous EHR programs. 

• Programs in the past lack adequate market research, requirements or 
technical agreement from respective communities. 

• Programs in the past lack of acquisition professionals and team to acquire 
the product. 

• Programs in the past had milestones that were not solidified. 

• Programs in the past lack appropriate buy-in from stakeholders. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM program? 

• Must develop the ability to train the users including plans for maintenance, 
testing and deployment of the system. 

• Local command governance. 

• Attempts to inhibit the commercial product with unrealistic requirements. 

Question 3) What major acquisition strategies were implemented to overcome 
these obstacles and risks? 

• Leveraging what has been done in the commercial environment. 

• Focused on commonalities of the uniformed health care system. 

• Did not create artificial requirements. 
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• Had functional representatives on team. 

Question 4) How would you describe the managerial strategies of PEO DHMS 
and the DHMSM PMO, and how did they contribute to the success of the DHMSM 
program? 

• Flatten the organization, reduce the amount of layers and bureaucracy for 
decisions to be made. 

• Develop meetings with a purpose. 

• Create relationships with requirements community. 

• Develop requirements that are static and stick to it. Allow for flexible with 
in the contract. 

• Grade industry on RFP sections F and M and not the entire PWS. 

• Feverage market research that had previous been done on the commercial 
product. 

Question 5) Based on your experiences as a member of the PEO DHMS team, 
what are the key lessons learned and advice you can share that can benefit DOD PM of 
other major software acquisition programs? 

• Setup a Health SYSCOM (System and Material Command). 

• A Health SYSCOM will have acquisition professionals which can rapidly 
capture technological advancements in a dynamic field. 

• Solidify what is the true requirement. 

• Feverage commonality, don’t highlight differences. 


Program Manager 2 
June 27, 2016 

Interview conducted via face-to-face 

Question 1) Were you involved in supporting any previous DOD EHR related 
programs? What do you believe were some of the key reasons for the failure of these past 
EHR programs? 

• Yes, during the iEHR. 
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• I don’t think DOD does development of these kind of software programs 
very well. It was good at one time during when we were writing code but 
now when there are commercial products. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM program? 

• From my perspective, when all this started, it was under the control of 
DCMO and DHA, [they were] involved in the process. The head of PEO 
came onboard PEO was established. Then it became a direct report to 
USD (AT&L) that is unlike anything else in DOD acquisition. What I 
think it also removed was DHA [because, they are] not an acquisition 
entity and it is not [their] core competency and they don’t do it well. 

Question 3) What major acquisition strategies were implemented to overcome 
these obstacles and risks? 

• I think the acquisition strategy was a tailored acquisition. Not all the 
normal documentation was required. 

• There was a great leadership emphasis to streamline the process, to get 
this thing moving fast. 

• Without the ability to tailor things, you will get drown in time, paper 
work, and bureaucracy. 

Question 4) How would you describe the managerial strategies of PEO DHMS 
and the DHMSM PMO, and how did they contribute to the success of the DHMSM 
program? 

• The original [PEO and DHMSM heads] were very hands on people. They 
looked to engaged the functional community directly and force people to 
make decisions and get on board. 

• [To] force people to have discussion on where the fraction points were and 
to overcome them. You do that by transparency and talking to people. 

• Taking the best of all that information and educating what the RFP would 
be and critique would be. Taking all that information and using it all for 
your benefit, I think it’s a key piece to the success. 

Question 5) Based on your experiences as a member of the PEO DHMS team, 
what are the key lessons learned and advice you can share that can benefit DOD PM of 
other major software acquisition programs? 

• Great leadership, clearly define leadership roles are very important. 
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• It needs to be an acquisition organization that runs the organization and 
not something inside of the functional community, meaning that the 
medical guy shouldn’t be the one in charge of the acquisition. 

• The functional [community] need[s] to be clearly involved and serve in 
that role. [They have] to tell you no kidding [what] they really need the 
[system] to do. 

• The functional [community] really needs to be force to define the 
requirements, the tighter you that, the better you’ll be able to buy it for 
them. The better the requirements, the better the acquisitions will be. 


Program Manager 3 
May 24, 2016 

Interview conducted via face-to-face 


Question 1) Were you involved in supporting any previous DOD EHR related 
programs? What do you believe were some of the key reasons for the failure of these past 
EHR programs? 

• Yes, for iEHR and Clinical Information Technology Program Office 
(CITPO). 

• DOD is a lousy software developer. 

• For iEHR, DOD attempted to develop and modify a product. They [DOD] 
are not coders or developers. 

• iEHR requirements were too problematic, too big of a product, two 
different secretariat organizations with different process and it was too big 
of a product. Which was probably a non-starter because of the two 
fundamentally different functions. Since process can’t be strategized, it 
can’t be implemented. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM program? 

• Congress hated us [DOD], due to the bad press about the failed EHR way 
ahead (2008-2009) program. 

• DOD had spent over 2.4 billion dollars for iEHR that never materialized 
from 2008-2013. 

• Lack of governance and lack of requirements. 
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• No requirements to put a contract action in place. 

• Lack of program office. Building a program office while attempting to 
start program and build strategy for ACAT I program. 

Question 3) What major acquisition strategies were implemented to overcome 
these obstacles and risks? 

• Being a direct report to the USD (AT&L) and USD Sectary of Defense 
and with a lot of leadership interest help reduce road blocks for a program 
that did not exist. Helped removed all barriers. 

• Reduced risk when placed under USD (AT&L) by removed obstacles, 
allow for aggressive strategy, test bed for new strategy (under USD 
[AT&L] guidance). Because staff is unlikely to disapprove previous 
approved USD (AT&L) guidance. 

• Highly tailored five fundamental documents instead of the 16 required 
documents for ACAT I programs (acquisition strategy, acquisition 
baseline, engineering master plan, test strategy, deployment change 
management plan). These five documents allowed for streamline process 
and in coordinated other strategy requirements. 

Question 4) How would you describe the managerial strategies of PEO DHMS 
and the DHMSM PMO, and how did they contribute to the success of the DHMSM 
program? 

• Complete transparency. 

• Said what we were going to do and we did it. 

• Aggressive scheduling with milestones. 

• Announced first industry day and held to the schedule and then published 
four other industry days which followed. 

• Published acquisition plans. 

• Meeting with companies/industry often. 

Question 5) Based on your experiences as a member of the PEO DHMS team, 
what are the key lessons learned and advice you can share that can benefit DOD PM of 
other major software acquisition programs? 

• Got to have 

1. Good requirements. 

2. Achievable and obtainable commercial contracting. 
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3. Accept commercial product, stable governance. 

4. Clear requirements from functional and technical side with enough 
trade space to allow for COTS environment. 

5. Allow enough space to leverage industry, be descriptive enough - 
to allow DOD to buy something. 

6. Solid acquisition strategy that everyone brought into such as PEO/ 
USD (AT&L)/services/functional sponsors. 

7. Cannot customize accepted commercial product. Must adopt 
COTS product with commercial capability. 
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APPENDIX C. PROGRAM INTERVIEWS—MANAGERS 


The following are excerpts from in-person and Email correspondence from 
managers. Personal identifiable information has been removed to protect the subjects’ 
privacy. 


Manager 1 
May 26, 2016 

Interview conducted via face-to-face 

Question 1) Were you involved in supporting any previous DOD EHR related programs? 
What do you believe were some of the key reasons for the failure of these past EHR 
programs? 

• Yes, iEHR increment 1 and 2. 

• The partnership between DOD and VA was not fully committed to by both 
parties. 

• While the program achieved acquisition decisions to move forward, VA 
withdrew because they only wanted to modernize their legacy system with 
DOD funding. 

• Requirements articulation with the Medical customer base is difficult. 

• Lack of empowered central leadership with the authority to make 
decisions for the enterprise. 

• Too many stakeholders reluctant to commit to a specific way forward, but 
also too many who can say no. Leadership by consensus does not work in 
a complex military machine. 

• Stakeholders are reluctant to commit because they might miss out on the 
latest technological offering. Many “leaders” have established themselves 
leveraging technology locally. Scaling a home grown solution is often not 
feasible or runs into the same challenges as a formal acquisition. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 

DHMSM PMO at the beginning of the DHMSM Program? 

• Lack of formal requirements. 

• Lack of empowered central leadership with the authority to make decision 
for the enterprise. 
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Question 3) What major acquisition strategies were implemented to overcome these 
obstacles and risks? 

• Third parties were tasked to make it happen. Congress to SECDEF, 
SECDEF to USD(AT&L) to the new PEO to the new PM. 

• Requirements development through draft RFPs and partnership with 
industry. 

• “Tailored” acquisition within the parameters of the DOD 5000.02 
(Authority to Proceed (ATP) vice traditional Milestone events). 

Question 4) How would you describe the managerial strategies of PEO DHMS and the 

DHMSM PMO, and how did they contribute to the success of the DHMSM Program? 

• Partnership and guidance from USD (AT&L), was they tailored the 
acquisition processes to deliver capability. 

Question 5) Based on your experiences as a member of the PEO DHMS team, what are 
the key lessons learned and advice you can share that can benefit DOD PM of other 
major software acquisition programs? 

• Force the organization to formalize requirements, to identify a central 
leader who is empowered to make decision for the enterprise - meaning 
all of DOD and four services, that health care normalize between the four 
service regardless of geographic location or operational mission, it’s about 
time that we normalize health care and truly make it purple - in sense 
making it purple and have influence about how the title 10 service execute 
their mission. 


Manager 2 
May 25, 2016 

Interview conducted via email exchange 

Question 1) Were you involved in supporting any previous DOD EHR related programs? 
What do you believe were some of the key reasons for the failure of these past EHR 
programs? 

• Yes. I have worked within the Tricare Management Activity and now 
DHA since 1998.1 don’t know that I would classify as failures. 

• In my opinion, there were delays due to the lack of a single empowered 
authority. There was not one vision with one way forward. Each Service 
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wanted to ensure respective processes and business rules were protected; 
not compromised. Each Service has requirements inherent to its mission. 

• CHCS was an early EHR program designed to assist the medical 
community at respective MTFs. Each desktop was configured based on 
need of the respective Service and MTF. Then along comes CHCS II 
(subsequently AHLTA). Although an improvement, AHLTA did not 
provide meet everyone’s expectations. CHCS could not simply be 
uploaded to the new system. 

• While AHLTA represents the largest DOD EHR capabilities in the world, 
it does not completely support all defined current needs and anticipated 
future needs. The existing capabilities fail to meet the vision of a 
comprehensive EHR and do not effectively support the presidential 
initiatives for an EHR or seamless data sharing with the necessary 
partners. The advancements in technology and changes in the complexity 
of medicine, healthcare delivery, dependencies on civilian managed care, 
data-sharing with the Federal and commercial partners and Operational 
Tempo require advanced EHR capabilities to meet current and future 
needs. 

• In my opinion, the lack of a single authoritative source, the design of the 
Government and Service Departments to have multiple layers of 
governance processes, long lead times to approve/obtain funding, sound 
working relationships between the functional (requirements) and technical 
communities along with the rotation of Service members led to significant 
delays or failures. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM Program? 

• Another obstacle included securing funding. There are many competing 
priorities for funding. The Government Program, Planning, Budgeting, 
and Execution System (PPBES) is designed to plan for many years of 
future funding requirements. This system was designed based on product 
development/manufacturing, deployment and sustainment. In this ever 
changing world of technology, the system is not flexible to keep up. 

• Another obstacle is staffing. First of all, a clear plan for staffing and 
manning the projects and programs is needed. Then obtaining the right 
talent, in a timely manner, to execute a plan that is filled with challenges 
ranging from long hiring time for civilians, Service members’ rotation 
schedules and competing requirements like the war, cost of contract 
support, and length of time to award contract are clearly obstacles in being 
able to accomplish the work. 

Question 3) What major acquisition strategies were implemented to overcome these 
obstacles and risks? 
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• From my perspective, the biggest obstacle was overcome when the PEO 
DHMS was stood up under USD (AT&L). This provided for an 
empowered authoritative source. 

• Staffing obstacles were overcome with the agreement between DHA and 
SPAWAR - allowing the PEO DHMS to hire skilled staff from the 
Working Capital Funds. 

• With the designation of ... [name redacted] as the PEO, [name redacted] 
rebuilt the credibility at the highest levels, Congress, AT&L, OMB, and 
the Services. This enabled the program to secure funding. 

• Rebuilding the broken relationships and ensuring the cost assessment and 
program evaluation was included as part of the process, enabled the Life 
Cycle Cost Estimate to be successfully developed/implemented. 
Additionally, via a “tailored” acquisition approach, all stakeholders were 
part of the process and included in the development of the DHMSM 
Acquisition Strategy 

Question 4) How would you describe the managerial strategies of PEO DHMS and the 
DHMSM PMO, and how did they contribute to the success of the DHMSM Program? 

• The years of acquisition experience enabled leadership to identify the 
processes inherently required for a successful COTS acquisition. They 
identified the right skills sets and staff needed to meeting the challenges of 
developing an RFP that clearly addressed and identified the requirements 
required for a vendor to provide a COTS solution. 

• They identified the right skills sets and staff needed to meeting the 
challenges of developing an RFP that clearly addressed and identified the 
requirements required for a vendor to provide a COTS solution. 

• Additionally, through their leadership, they ensured the right communities 
were part of the evaluation process to ensure the functional requirements 
would be met. In addition to their leadership, folks were highly motivated 
to support the mission of obtaining a cradle-to-grave EHR for our Service 
Members, Families and Beneficiaries. 

Question 5) Based on your experiences as a member of the PEO DHMS team, what are 
the key lessons learned and advice you can share that can benefit DOD PM of other 
major software acquisition programs? 

• Teamwork and Communication 

• Clear understanding of requirements and desired outcomes 

• Identify stakeholders and beneficiaries - make them part of the process 

• Create positive working environments that motivate employees 
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• Establish realistic goals and timelines 

Manager 3 
May 31, 2016 

Interview conducted via face-to-face 

Question 1) Were you involved in supporting any previous DOD EHR related programs? 
What do you believe were some of the key reasons for the failure of these past EHR 
programs? 

• Yes. 

• This was a time when DOD was not ready to do this but we still went 

through a whole process with the DHMSM. The IEHR was trying to do 

that with us and do it with the VA, and VA was less prepared to deal with 
standardization and process flows than then DOD was. The DOD had 
three services Navy, Army, Air Force, so the DOD has three different 
policy things. VA has 120 modules of VISTA. They have to decentralize 
what they do, they have medical centers that run their own VISTA and has 
its own work flows i.e., billing. Before DOD and VA can work together, 
you have to have three [services] in DOD and 120 [modules] on VA. If 
you can’t get together what system your building and variables that you 
will put in play, then you’re sunk. 

• There was a huge group at the VA that wanted to re-litigate the decision 
for the DOD not to buy into the VISTA system. The VA decision was, that 
we think we should use Vista as the basis for the EHR, DOD that doesn’t 
make sense to us, we don’t have Vista programmers and will cost us more 
to use VISTA. The VA recognize that DOD as a partner and a cash source. 
The smaller department will leverage the bigger department, so they can 
pay for it, is what VA wanted. You have backers of VA that DOD is 
leaving VA out. Some didn’t understand interoperability and some knew it 
but understood [that] VA would be better with DOD. 

• When VISTA came out, it was state of the art but industry has past that. 

• The DOD came to the conclusion earlier than the VA, because the DOD 
experience with buying all kinds of other weapons systems. We don’t 
design airplane anymore, we buy planes from Lockheed and Boeing 
designs. There was a time when we designed our own airplanes and over 
saw the building of them. We outsource those professionals decades ago. 
And the VA still have VISTA programmers on its staff. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 

DHMSM PMO at the beginning of the DHMSM program? 
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• When DHMSM was created, you had a medical community that felt like 
they have their best and brightest in endeavor taken away from them by 
failures that happen in the past. And by letting the acquisition people 
procure the EHR—what do they know about EHR? There were hurt 
feelings throughout the medical community. That was an obstacle. 

• Services fighting one another and fighting with DHA to have some of 
governance and decision making process. You had three medical systems 
that operated with autonomy for decades and now they are being forced 
into one central organization. 

Question 3) What major acquisition strategies were implemented to overcome these 
obstacles and risks? 

• [The Head of PEO] said that he was in the DOD but not in the medical 
community. The DOD is in the buying business. Our focus is not as a 
health care organization and we need to leave the best practices of the 
health care organization to the industry. We need to decide if we can pick 
up those best practices so we don’t have to be responsible for creating 
them. 

• Taking away the personal ownership of those most aligned with that 
system, that lead to a better decision. 

Question 4) How would you describe the managerial strategies of PEO DHMS and the 
DHMSM PMO, and how did they contribute to the success of the DHMSM program? 

• N/A 

Question 5) Based on your experiences as a member of the PEO DHMS team, what are 
the key lessons learned and advice you can share that can benefit DOD PM of other 
major software acquisition programs? 

• We need to have a medical health system that is able to leverage the best 
practices of the medical community. And adopt their best practice so the 
medical community can be at the forefront of the industry. DOD made a 
transition by bring in an acquisition force in that wanted to acquire or get 
the best electronic health record, it takes all the personal feeling out of it. 

• Admin and bureaucracy and policy vice technology and that’s [the] best 
thing that you can do for any IT or weapons system is [to] ensure that the 
customer (biggest stake) is removed from the acquisition decision and 
leave the business decision to acquisition people. 
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Manager 4 
May 17, 2016 

Interview conducted via face-to-face 

Question 1) Were you involved in supporting any previous DOD EHR related programs? 
What do you believe were some of the key reasons for the failure of these past EHR 
programs? 

• I was not involved in previous programs. 

Question 2) What were the key obstacles and risks faced by PEO DHMS and the 
DHMSM PMO at the beginning of the DHMSM program? 

• I believe that perceptions from the previous implementations affected the 
program because ... no one took the project seriously to get a contract in 
place. 

• It required collaboration across multiple organizations which requires 
attention to ensure collaboration. 

• There were aggressive schedule constraints due in part to non-execution of 
previous programs. 

Question 3) What major acquisition strategies were implemented to overcome these 
obstacles and risks? 

• Not sure it is an acquisition strategy but the use of common sense to not 
allow stupid things get in the way. 

• Leadership not afraid to challenge status quo to move things forward if 
there is a road block. 

Question 4) How would you describe the managerial strategies of PEO DHMS and the 
DHMSM PMO, and how did they contribute to the success of the DHMSM Program? 

• They weren’t afraid to take risks. 

• They did not wait to have 100 percent of items complete before they 
would execute. For example, as information became available, [the] draft 
RFP were released on multiple occasions to get industry involved early. 

Question 5) Based on your experiences as a member of the PEO DHMS team, what are 

the key lessons learned and advice you can share that can benefit DOD PM of other 

major software acquisition programs? 
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• Do not be afraid to take risks. 

• Encourage collaboration to include frequent and early interaction. 

• Do not be afraid to ask or recommend to streamlining the acquisition 
process. 
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